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(57) Abstract: Systems and methods are provided for decoding a message symbol in an audio signal. This message symbol is 
represented by first and second code symbols displaced in time. Values representing the code signals are accumulated (116) and the 
accumulated values are examined to detect the message symbol (1 20). 
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DECODING OF 
INFORMATION IN AUDIO SIGNALS 



BACKGROUND OF THE INVENTION 

The present invention relates to methods and apparatus for extracting an 
information signal from an encoded audio signal. 

There are various motivations to permanently or indelibly incorporate 
information signals into audio signals, referred to as ts watermarking." Such an audio 

5 watermark may provide, for example, an indication of authorship, content, lineage, 
existence of copyright, or the like for the audio signals so marked. Alternatively, 
other information may be incorporated into audio signals either concerning the signal 
itself or unrelated to it. The information may be incorporated in an audio signal for 
various purposes, such as identification or as an address or command, whether or not 

10 related to the signal itself. 

There is considerable interest in encoding audio signals with information to 
produce encoded audio signals having substantially the same perceptible 
characteristics as the original unehcoded audio signals. Recent successful techniques 
exploit the psychoacoustic masking effect of the human auditory system whereby 

1 5 certain sounds are humanly imperceptible when received along with other sounds. 

One particularly successful utilization of the psychoacoustic masking effect is 
described in U.S. Patents No. 5,450,490 and No. 5,764,763 (Jensen et al.) in which 
information is represented by a multiple-frequency code signal which is incorporated 
into an audio signal based upon the masking ability of the audio signal. The encoded 

20 audio signal is suitable for broadcast transmission and reception as well as for 
recording and reproduction. When received the audio signal is then processed to 
detect the presence of the multiple-frequency code signal. Sometimes, only a portion 
of the multiple-frequency code signal, e.g., a number of single frequency code 
components, inserted into the original audio signal are detected in the received audio 

25 signal. If a sufficient quantity of code components is detected, the information signal 
itself may be recovered. 
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Generally, an acoustic signal having low amplitude levels will have only 
minimal capacity, if any at all, to acoustically mask an information signal. For 
example, such low amplitude levels can occur during a pause in a conversation, 
during an interlude between segments of music, or even within certain types of music. 

5 During a lengthy period of low amplitude levels, it may be difficult to incorporate a 
code signal in an audio signal without causing the encoded audio signal to differ from 
the original in an acoustically perceptible manner. 

A further problem is the occurrence of burst errors during the transmission or 
reproduction of encoded audio signals. Burst errors may appear as temporally 

10 contiguous segments of signal error. Such errors generally are unpredictable and 

substantially affect the content of an encoded audio signal. Burst errors typically arise 
from failure in a transmission channel or reproduction device due to severe external 
interferences, such as an overlapping of signals from different transmission channels, 
an occurrence of system power spikes, an interruption in normal operations, an 

15 introduction of noise contamination (intentionally or otherwise), and the like. In a 
transmission system, such circumstances may cause a portion of the transmitted 
encoded audio signals to be entirely unreceivable or significantly altered. Absent 
retransmission of the encoded audio signal, the affected portion of the encoded audio 
may be wholly unrecoverable, while in other instances alterations to the encoded 

20 audio signal may render the embedded information signal undetectable. In many 
applications, such as radio and television broadcasting, real-time retransmission of 
encoded audio signals is simply unfeasible. 

In systems for acoustically reproducing audio signals recorded on media, a 
variety of factors may cause burst errors in the reproduced acoustic signal. 

25 Commonly, an irregularity in the recording media, caused by damage, obstruction, or 
wear, results in certain portions of recorded audio signals being unreproducable or 
significantly altered upon reproduction. Also, misalignment of or interference with 
the recording or reproducing mechanism relative to the recording medium can cause 
burst-type errors during an acoustic reproduction of recorded audio signals. Further, 

30 the acoustic limitations of a speaker as well as the acoustic characteristics of the 

listening environment may result in spatial irregularities in the distribution of acoustic 
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energy. Such irregularities may cause burst errors to occur in received acoustic 
signals, interfering with code recovery. 

OBJECTS AND SUMMARY OF THE INVENTION 

Therefore, an object of the present invention is to provide systems and 
5 methods for detecting code symbols in audio signals which alleviate the problems 
caused by periods of low signal levels and burst errors. 

It is another object of the invention to provide such systems and methods 
which afford reliable operation under adverse conditions. 

It is a further object of the invention to provide such systems and methods 
1 0 which are robust. 

In accordance with an aspect of the present invention, systems and methods 
are provided for decoding at least one message symbol represented by a plurality of 
code symbols in an audio signal. The systems and methods comprise the means for 
and the steps of, respectively, receiving first and second code symbols representing a 
1 5 common message symbol, the first and second code symbols being displaced in time 
in the audio signal, accumulating a first signal value representing the first code 
symbol and a second signal value representing the second code symbol, and 
examining the accumulated first and second signal values to detect the common 
message symbol. 

20 In accordance with another aspect of the present invention, a system is 

provided for decoding at least one message symbol represented by a plurality of code 
symbols in an audio signal. The system comprises, an input device for receiving first 
and second code symbols representing a common message symbol, the first and 
second code symbols being displaced in time in the audio signal; and a digital 

25 processor in communication with the input device to receive data therefrom 
representing the first and second code symbols, the digital processor being 
programmed to accumulate a first signal value representing the first code symbol and 
a second signal value representing the second code symbol, the digital processor 
being further programmed to examine the accumulated first and second signal values 

30 to detect the common message symbol. 
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In certain embodiments, the first and second signal values are accumulated by 
storing the values separately and the common message symbol is detected by 
examining both of the separately stored values. The first and second signal values 
may represent signal values derived from multiple other signal values, such as values 

5 of individual code frequency components, or a single signal value, such as a measure 
of the magnitude of a single code frequency component. Moreover, a derived value 
may be obtained as a linear combination of multiple signal values, such as a 
summation of weighted or unweighted values, or as a non-linear function thereof. 

In further embodiments, the first and second signal values are accumulated by 

10 producing a third signal value derived from the first and second values. The third 

signal value in some embodiments is derived through a linear combination of the first 
and second signal values, such as a weighted or unweighted summation thereof, or as 
a nonlinear function thereof. 

Other objects, features, and advantages according to the present invention will 

15 become apparent from the following detailed description of certain advantageous 

embodiments when read in conjunction with the accompanying drawings in which the 
same components are identified by the same reference numerals. 

BRIEF DESCRIPTION OF THE DRAWINGS 

Figure 1 is a functional block diagram of an encoding apparatus; 
20 Figure 2 is a table to which reference will be made in explaining a 

methodology for encoding information in an audio signal; 

Figures 3 A, 3B, and 3C are schematic diagrams which illustrate an audio 
signal encoding methodology; 

Figure 4 is another table to which reference will be made in explaining a 
25 methodology for encoding information in an audio signal; 

Figure 5 is a block diagram illustrating a multiple-stage audio signal encoding 

system; 

Figure 6 is a functional block diagram of a personal portable meter; 
Figure 7 is a functional block diagram illustrating a decoding apparatus; 
30 Figure 8 is a flowchart illustrating a methodology for retrieving an information 

code from an encoded audio signal; 
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Figure 9 is a schematic diagram of a circular SNR buffer used in carrying out 
the methodology of Figure 8; 

Figure 10 is a flow chart illustrating another methodology for retrieving an 
information code from an encoded audio signal. 

DETAILED DESCRIPTION OF CERTAIN ADVANTAGEOUS 

EMBODIMENTS 

The present invention relates to the use of especially robust encoding which 

converts information into redundant sequences of code symbols. In certain 

embodiments, each code symbol is represented by a set of different, predetermined 

single-frequency code signals; however, in other embodiments different code symbols 

may optionally share certain single-frequency code signals or may be provided by a 

methodology which does not assign predetermined frequency components to a given 

symbol. The redundant sequence of symbols is incorporated into the audio signals to 

produce encoded audio signals that are unnoticed by the listener but nevertheless 

recoverable. 

The redundant code symbol sequence is especially suited for incorporation 
into audio signals having low masking capacity, such as audio signals having many 
low amplitude portions or the like. Additionally, when incorporated into audio 
signals, the redundant sequence of code symbols resists degradation by burst errors 
which affect temporally contiguous audio signals. As described hereinabove, such 
errors may be the result of imperfect audio signal recording, reproduction, and/or 
storage processes, transmission of the audio signals through a lossy and/or noisy 
channel, irregularities in an acoustic environment, or the like. 

To recover the encoded information in certain advantageous embodiments, the 
encoded audio signals are examined in an attempt to detect the presence of 
predetermined single-frequency code components. During the encoding process, 
some single- frequency code components may not have been incorporated into the 
audio signals in certain signal intervals due to insufficient masking capacity in the 
audio signals in these intervals. Burst errors which have corrupted portions of the 
encoded audio signals can result in the deletion of certain code signals from the 
encoded audio signals or in the insertion of erroneous signals, such as noise, into the 
encoded audio signals. Thus, examination of the encoded audio signals is likely to 
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reveal a much distorted version of the original sequence of sets of single-frequency 
code signals that represented the information. 

The single-frequency code components that are recovered, along with the 
erroneous additional signals that are mistakenly detected as code signals, are 
5 processed to discern the original sequence of code symbols, if possible. The code 
signal detection and processing operations are specifically adapted to exploit the 
strengths of the encoding methodology. As a result, the detection and processing 
methodology of the present invention provides improved error tolerance. 

Figure 1 is a functional block diagram of an audio signal encoder 1 0. Encoder 

10 10 implements an optional symbol generation function 12, a symbol sequence 

generation function 14, a symbol encoding function 16, an acoustic masking effect 
evaluation/adjustment function 18, and an audio signal inclusion function 20. 
Preferably encoder 10 comprises a software-controlled computer system. The 
computer may be provided with an analog processor for sampling an analog audio 

15 signal to be encoded, or may input the audio signal directly in digital form, with or 
without resampling. Alternatively, encoder 10 may comprise one or more discrete 
signal processing components. 

The symbol generation function 12, when employed, translates an information 
signal into a set of code symbols. This function may be carried out with the use of a 

20 memory device, such as a semiconductor EPROM of the computer system, which is 
prestored with a table of code symbols suitable for indexing with respect to an 
information signal. An example of a table for translating an information signal into a 
code symbol for certain applications is shown in Figure 2. The table may be stored on 
a hard drive or other suitable storage device of the computer system. The symbol 

25 generation function may also be carried out by one or more discrete components, such 
as an EPROM and associated control devices, by a logic array, by an application 
specific integrated circuit, or any other suitable device or combination of devices. 
The symbol generation function may also be implemented by one or more devices 
which also implement one or more of the remaining functions illustrated in Figure 1 . 

30 The symbol sequence generating function 14 formats the symbols produced by 

the symbol generating function (or input directly to the encoder 10) into a redundant 
sequence of code or information symbols. As part of the formatting process, in 
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certain embodiments marker and/or synchronization symbols are added to the 
sequence of code symbols. The redundant sequence of code symbols is designed to 
be especially resistant to burst errors and audio signal encoding processes. Further 
explanation of redundant sequences of code symbols in accordance with certain 

5 embodiments will be provided in connection with the discussion of Figures 3 A, 3B, 
and 3C below. Preferably, the generating function 14 is implemented in a processing 
device, such as a microprocessor system, or by a dedicated formatting device, such as 
an application specific integrated circuit or a logic array, by a plurality of components 
or a combination of the foregoing. The symbol sequence generating function may 

1 0 also be implemented by one or more devices which also implement one or more of the 
remaining functions illustrated in Figure 1. 

As noted above, the symbol sequence generating function 14 is optional. For 
example, the encoding process may be carried out such that the information signal is 
translated directly into a predetermined symbol sequence, without implementing 

1 5 separate symbol generating and symbol sequence generating functions. 

Each symbol of the sequence of symbols thus produced is converted by the 
symbol encoding function 16 into a plurality of single-frequency code signals. In 
certain advantageous embodiments the symbol encoding function is performed by 
means of a memory device of the computer system, such as a semiconductor EPROM, 

20 which is prestored with sets of single-frequency code signals that correspond to each 
symbol. An example of a table of symbols and corresponding sets of single- 
frequency code signals is shown in Figure 4. 

Alternatively, the sets of code signals may be stored on a hard drive or other 
suitable storage device of the computer system. The encoding function may also be 

25 implemented by one or more discrete components, such as an EPROM and associated 
control devices, by a logic array, by an application specific integrated circuit or any 
other suitable device or combination of devices. The encoding function may also be 
carried out by one or more devices which also implement one or more of the 
remaining functions illustrated in Figure 1. 

30 In the alternative, the encoded sequence may be generated directly from the 

information signal, without implementing the separate functions 12, 14, and 16. 
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The acoustic masking effect evaluation/adjustment function 18 determines the 
capacity of an input audio signal to mask single-frequency code signals produced by 
the symbol encoding function 16. Based upon a determination of the masking ability 
of the audio signal, the function 18 generates adjustment parameters to adjust the 

5 relative magnitudes of the single-frequency code signals so that such code signals will 
be rendered inaudible by a human listener when incorporated into the audio signal. 
Where the audio signal is determined to have low masking capacity, due to low signal 
amplitude or other signal characteristics, the adjustment parameters may reduce the 
magnitudes of certain code signals to extremely-low levels or may nullify such signals 

1 0 entirely. Conversely, where the audio signal is determined to have a greater masking 
capacity, such capacity may be utilized through the generation of adjustment 
parameters that increase the magnitudes of particular code signals. Code signals 
having increased magnitudes are generally more likely to be distinguishable from 
noise and thus detectable by a decoding device. Further details of certain 

15 advantageous embodiments of such evaluation/adjustment function are set forth in 
U.S. Patents No. 5,764,763 and No. 5,450,490 to Jensen, et al., each entitled 
Apparatus and Methods for Including Codes in Audio Signals and Decoding, which 
are incorporated herein by reference in their entirety. 

In certain embodiments, the function 18 applies the adjustment parameters to 

20 the single-frequency code signals to produce adjusted single-frequency code signals. 
The adjusted code signals are included in the audio signal by the function 20. 
Alternatively, the function 18 supplies the adjustment parameters along with the 
single-frequency code signals for adjustment and inclusion in the audio signal by the 
function 20. In still other embodiments, the function 18 is combined with one or 

25 more of the functions 12, 14, and 16 to produce magnitude-adjusted single-frequency 
code signals directly. 

In certain embodiments, the acoustic masking effect evaluation/adjustment 
function 18 is implemented in a processing device, such as a microprocessor system 
which may also implement one or more of the additional functions illustrated in 

30 Figure 1 . The function 18 may also be earned out by a dedicated device, such as an 
application specific integrated circuit or a logic array, or by a plurality of discrete 
components, or a combination of the foregoing. 
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The code inclusion function 20 combines the single-frequency code 
components with the audio signal to produce an encoded audio signal. In a 
straightforward implementation, the function 20 simply adds the single-frequency 
code signals directly to the audio signal. However, the function 20 may overlay the 
5 code signals upon the audio signal. Alternatively, modulator 20 may modify the 
amplitudes of frequencies within the audio signal according to an input from acoustic 
masking effect evaluation function 1 8 to produce an encoded audio signal that 
includes the adjusted code signals. Moreover, the code inclusion function may be 
carried out either in the time domain or in the frequency domain. The code inclusion 
1 0 function 20 may be implemented by means of an adding circuit, or by means of a 
processor. This function may also be implemented by one or more devices described 
above which also implement one or more of the remaining functions illustrated in 
Figure 1. 

One or more of the functions 12 through 20 may be implemented by a single 

15 device. In certain advantageous embodiments, the functions 12, 14, 16 and 18 are 

implemented by a single processor, and in still others a single processor carries out all 
of the functions illustrated in Figure 1. Moreover, two or more of the functions 12, 
14, 16 and 18 may be implemented by means of a single table maintained in an 
appropriate storage device. 

20 Figure 2 illustrates an exemplary translation table for converting an 

information signal into a code symbol. As shown, an information signal may 
comprise information regarding the content of, the characteristics of, or other 
considerations relating to a particular audio signal. For example, it is contemplated 
that an audio signal could be modified to include an inaudible indication that 

25 copyright is claimed in the audio program. Correspondingly, a symbol, such as Si, 
may be utilized to indicate that copyright is claimed in the particular work. Similarly, 
an author may be identified with a unique symbol S2 or a broadcasting station 
identified with a unique symbol S3. Further, a particular date could be represented by 
a symbol S4. Of course many other types of information could be included in an 

30 information signal and translated into a symbol. For example, information such as 
addresses, commands, encryption keys, etc., may be encoded in such symbols. 
Alternatively, sets or sequences of symbols, in addition to or instead of individual 
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symbols, may be utilized to represent particular types of information. As another 
alternative, an entire symbolic language may be implemented to represent any type of 
information signal. Also, the encoded information need not be related to the audio 
signal. 

5 Figure 3 A is a schematic diagram illustrating a stream of symbols which might 

be generated by the symbol generating function 12 of Figure 1, while Figures 3B and 
3C are schematic diagrams illustrating sequences of symbols which might be 
generated by the symbol sequence generating function 14 of Figure 1 in response to 
the symbol stream of Figure 3 A. In Figures 3 A through 3C, Si, S 2 , S3, and S4 are 

10 used as examples of symbols to illustrate features of the present invention and are not 
meant to limit its applicability. For example, the information represented by any one 
or more of the symbols S], S 2 , S3 or S 4 may be selected arbitrarily without regard to 
that represented by any one or more of the other symbols. 

Figure 3B illustrates an example of a core unit of a redundant symbol 

15 sequence representative of an input set of four symbols, Si, S2, S3, and S4. The core 
unit begins with a first message segment having sequence or marker symbol, Sa, 
followed by the four input data symbols, followed by three repeating message 
segments each comprised of a sequence or marker symbol, Sb, and the four input 
symbols. For many applications, this core unit alone is sufficiently redundant to 

20 provide the required level of survivability. Alternatively, this core unit may itself be 
repeated to increase survivability. Moreover, the core unit may have more or less 
than four message segments, as well as segments having more or less than four or five 
symbols. 

Generalizing from this example, an input set of N symbols, Si, S2, S3,..., Sn-i, 
25 S N , is represented by the redundant symbol sequence comprising Sa, Si, S2, S3,...S N .i, 
Sn, followed by (P-l) repeating segments comprising Sb, Si, S2, S3,...Sn-i, Sn. As in 
the example, this core unit may itself be repeated to increase survivability. In 
addition, the sequence of symbols in the message segments may be varied from 
segment to segment so long as the decoder is arranged to recognize corresponding 
30 symbols in the various segments. Moreover, different sequence or marker symbols 
and combinations thereof may be employed, and the positions of the markers with 
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respect to the data symbols may be arranged differently. For example, the sequence 
can take the form, Si, S2, Sa» Sn or the form, Si, S2, Sn, Sa. 

Figure 3C illustrates an example of an advantageous core unit of a redundant 
symbol sequence representative of an input set of four data symbols, S|, S2, S3, and 

5 S 4 . The core unit begins with a sequence or marker symbol, S A , followed by the four 
input data symbols, followed by a sequence or marker symbol, Sb, followed by S(i+6) 
mod m, S(2+6) mod m, S(3+6) mod m, S(4+6) mod m> where M is the number of different symbols 
in the available symbol set and where 5 is an offset having a value between 0 and M. 
In an advantageous embodiment, the offset 8 is selected as a CRC checksum. In still 

10 other embodiments, the value of the offset 8 is varied from time to time to encode 
additional information in the message. For example, if the offset can vary from 0 to 
9, nine different information states can be encoded in the offset. 

Generalizing from this example, an input set of N symbols, Si, S 2 , S3,.-.Sn-i, 
S n , is represented by the redundant symbol sequence comprising S A) Si, S2, S3,.»Sn-i, 

15 Sn, Sb, S(i+8)modM> S(2+6)modM, S(3+5)modM» ».S(N-l+5) mod M, S(N+5)modM- That is, the 

same information is represented by two or more different symbols in the same core 
unit and recognized according to their order therein. In addition, these core units may 
themselves be repeated to increase survivability. Since the same information is 
represented by multiple different symbols, the coding is made substantially more 

20 robust. For example, the structure of an audio signal can mimic the frequency 

component of one of the data symbols Sn, but the likelihood that the audio signal will 
also mimic its corresponding offset S(n+ 8) mod m at its predetermined occurrence is very 
much lower. Also, since the offset is the same for all symbols within a given 
segment, this information provides a further check on the validity of the detected 

25 symbols within that segment. Consequently, the encoding format of Figure 3C 

substantially reduces the likelihood of false detections induced by the structure of the 
audio signal. 

A particular strength of the redundant sequence exemplified in Figure 3 is its 
utilization of the input symbols in their original order followed by (a) a different 
30 arrangement of the input symbols, (b) an arrangement of symbols that includes other 
symbols in place of one or more of the input symbols, with or without rearrangement 
of input symbol order, or (c) an arrangement of symbols different from the input 
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symbols. Arrangements (b) and (c) are particularly robust since, upon symbol 
encoding, an increased diversity of single-frequency code signals is achieved. 
Assuming that the input symbols are encoded collectively from among a first group of 
code signals, symbols in arrangements (b) and (c) will be encoded with another group 
5 of code signals which to some extent does not overlap the first group. A greater 

diversity of code signals will generally increase the likelihood that some code signals 
are within the masking capacity of the audio signal. 

The table of Figure 4 illustrates an exemplary conversion for a sequence or 
marker symbol, Sa, a sequence or marker symbol, S B , and N data symbols, Si, S2, 

10 S3,..., Sn-i, Sn, into corresponding sets of M single-frequency code signals f lx , f2 X , 

f3 X ,..., f[M-i]x, fMx, where x references the identifying subscript of the particular symbol. 
Although the single-frequency code signals may occur throughout the frequency 
range of the audio signal and, to some extent, outside such frequency range, the code 
signals of this embodiment are within the frequency range 500 Hz to 5500 Hz but 

15 may be selected as a different frequency range. In one embodiment, the sets of M 
single-frequency code signals may share certain single- frequency code signals; 
however, in a preferred embodiment, the single-frequency code signals are completely 
non-overlapping. Moreover, it is not necessary that all symbols be represented by the 
same number of frequency components. 

20 Figure 5 illustrates a multiple-stage audio signal encoding system 50. This 

system implements multiple audio signal encoders to successively encode an audio 
signal 52 as it travels along a typical audio signal distribution network. At each stage 
of distribution, the audio signal is successively encoded with an information signal 
pertinent to the particular stage. Preferably, the successive encodings of the 

25 respective information signals do not produce code signals that overlap in frequency. 
Nevertheless, due to the robust nature of the encoding methodology, partial overlap 
among the frequency components of the respective encoded information signals is 
tolerable. System 50 includes a recording facility 54; a broadcaster 66; a relay station 
76; audio signal encoders 58, 70, and 80; an audio signal recorder 62; a listener 

30 facility 86; and an audio signal decoder 88. 

Recording facility 54 includes apparatus for receiving and encoding audio 
signals and recording encoded audio signals upon a storage medium. Specifically, 
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facility 54 includes audio signal encoder 58 and audio signal recorder 62. Audio 
signal encoder 58 receives an audio signal feed 52 and a recording information signal 
56 and encodes audio signal 52 with information signal 56 to produce an encoded 
audio signal 60. Audio signal feed 52 may be produced by any conventional source 

5 of audio signals such as a microphone, an apparatus for reproducing recorded audio 
signals, or the like. Recording information signal 56 preferably comprises 
information regarding audio signal feed 52, such as its authorship, content, or lineage, 
or the existence of copyright, or the like. Alternatively, recording information signal 
56 may comprise any type of data. 

10 Recorder 62 is a conventional device for recording encoded audio signals 60 

upon a storage medium which is suitable for distribution to one or more broadcasters 
66. Alternatively, audio signal recorder 62 may be omitted entirely. Encoded audio 
signals 60 may be distributed via distribution of the recorded storage media or via a 
communication link 64. Communication link 64 extends between recording facility 

15 54 and broadcaster 66 and may comprise a broadcast channel, a microwave link, a 
wire or fiber optic connection, or the like. 

Broadcaster 66 is a broadcasting station that receives encoded audio signals 
60, further encodes such signals 60 with a broadcaster information signal 68 to 
produce a twice-encoded audio signal 72, and broadcasts the twice-encoded audio 

20 signal 72 along a transmission path 74. Broadcaster 66 includes an audio signal 

encoder 70 which receives encoded audio signal 60 from recording facility 54 and a 
broadcaster information signal 68. Broadcaster information signal 68 may comprise 
information regarding broadcaster 66, such as an identification code, or regarding the 
broadcasting process, such as the time, date or characteristics of the broadcast, the 

25 intended recipient(s) of the broadcast signal, or the like. Encoder 70 encodes encoded 
audio signal 60 with information signal 68 to produce twice-encoded audio signal 72. 
Transmission path 74 extends between broadcaster 66 and relay station 76 may 
comprise a broadcast channel, a microwave link, a wire or fiber optic connection, or 
the like. 

30 Relay station 76 receives a twice-encoded audio signal 72 from broadcaster 

66, further encodes that signal with a relay station information signal 78, and 
transmits the thrice-encoded audio signal 82 to a listener facility 86 via a transmission 
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path 84. Relay station 76 includes an audio signal encoder 80 which receives twice- 
encoded audio signal 72 from broadcaster 66 and a relay station information signal 
78. Relay station information signal 78 preferably comprises information regarding 
relay station 76, such as an identification code, or regarding the process of relaying 

5 the broadcast signal, such as the time, date or characteristics of the relay, the intended 
recipient(s) of the relayed signal, or the like. Encoder 80 encodes twice-encoded 
audio signal 72 with relay station information signal 78 to produce thrice-encoded 
audio signal 82. Transmission path 84 extends between relay station 76 and listener 
facility 86 and may comprise a broadcast channel, a microwave link, a wire or fiber 

10 optic connection, or the like. Optionally, transmission path 84 may be an acoustic 
transmission path. 

Listener facility 86 receives thrice-encoded audio signal 82 from relay station 
76. In audience estimate applications, listener facility 86 is located where a human 
listener may perceive an acoustic reproduction of audio signal 82. If audio signal 82 

15 is transmitted as an electromagnetic signal, listener facility 86 preferably includes a 
device for acoustically reproducing that signal for the human listener. However, if 
audio signal 82 is stored upon a storage medium, listener facility 86 preferably 
includes a device for reproducing signal 82 from the storage medium. 

In other applications, such as music identification and commercial monitoring, 

20 a monitoring facility is employed rather than listener 86. In such a monitoring 

facility, the audio signal 82 preferably is processed to receive the encoded message 
without acoustic reproduction. 

Audio signal decoder 88 may receive thrice encoded audio signal 82 as an 
audio signal or, optionally, as an acoustic signal. Decoder 88 decodes audio signal 82 

25 to recover one or more of the information signals encoded therein. Preferably, the 
recovered information signal(s) are processed at listener facility 86 or recorded on a 
storage medium for later processing. 

Alternatively, the recovered information signal(s) may be converted into 
images for visual display to the listener. 

30 In an alternate embodiment, recording facility 54 is omitted from system 50. 

Audio signal feed 52, representing, for example, a live audio performance, is provided 
directly to broadcaster 66 for encoding and broadcast. Accordingly, broadcaster 



WO 00/72309 PCT/US00/14057 

15 

information signal 68 may further comprise information regarding audio signal feed 
52, such as its authorship, content, or lineage, or the existence of copyright, or the 
like. 

In another alternate embodiment, relay station 76 is omitted from system 50. 
5 Broadcaster 66 provides twice-encoded audio signal 72 directly to listener 86 via 
transmission path 74 which is modified to extend therebetween. As a further 
alternative, both recording facility 54 and relay station 76 may be omitted from 
system 50. 

In another alternate embodiment, broadcaster 66 and relay station 76 are 
10 omitted from system 50. Optionally, communication link 64 is modified to extend 
between recording facility 54 and listener facility 86 and to carry encoded audio 
signal 60 therebetween. Preferably, audio signal recorder 62 records encoded audio 
signal 60 upon a storage medium which is thereafter conveyed to listener facility 86. 
An optional reproduction device at listener facility 86 reproduces the encoded audio 
1 5 signal from the storage medium for decoding and/or acoustic reproduction. 

Figure 6 provides an example of a personal portable meter 40 for use in 
audience estimate applications. The meter 90 includes a housing 92, illustrated in 
phantom lines, having a size and shape permitting it to be carried on the person of an 
audience member. For example, the housing may have the same size and shape as a 
20 pager unit. 

A microphone 93 is within the housing 92 and serves as an acoustic transducer 
to transduce received acoustic energy, including encoded audio signals, to analog 
electrical signals. The analog signals are converted to digital by an analog to digital 
converter and the digital signals are then supplied to a digital signal processor (DSP) 

25 95. The DSP 95 implements a decoder in accordance with the present invention in 
order to detect the presence of predetermined codes in the audio energy received by 
the microphone 93 indicating that the person carrying the personal portable meter 90 
has been exposed to a broadcast of a certain station or channel. If so, the DSP 95 
stores a signal representing such detection in its internal memory along with an 

30 associated time signal. 

The meter 90 also includes a data transmitter/receiver, such as an infrared 
transmitter/receiver 97 coupled with the DSP 95. The transmitter/receiver 97 enables 
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the DSP 95 to provide its data to a facility for processing such data from multiple 
meters 90 to produce audience estimates, as well as to receive instructions and data, 
for example, to set up the meter 90 for carrying out a new audience survey. 

Decoders in accordance with certain advantageous embodiments of the present 

5 invention are illustrated by the functional block diagram of Figure 7. An audio signal 
which may be encoded as described hereinabove with a plurality of code symbols, is 
received at an input 102. The received audio signal may be a broadcast, internet or 
otherwise communicated signal, or a reproduced signal. It may be a direct coupled or 
an acoustically coupled signal. From the following description in connection with the 

10 accompanying drawings, it will be appreciated that the decoder 100 is capable of 
detecting codes in addition to those arranged in the formats disclosed hereinabove. 

For received audio signals in the time domain, the decoder 100 transforms 
such signals to the frequency domain by means of a function 106. The function 106 
preferably is performed by a digital processor implementing a fast Fourier transform 

1 5 (FFT) although a direct cosine transform, a chirp transform or a Winograd transform 
algorithm (WFTA) may be employed in the alternative. Any other 
time-to-frequency-domain transformation function providing the necessary resolution 
may be employed in place of these. It will be appreciated that in certain 
implementations, the function 106 may also be carried out by analog or digital filters, 

20 by an application specific integrated circuit, or any other suitable device or 

combination of devices. The function 106 may also be implemented by one or more 
devices which also implement one or more of the remaining functions illustrated in 
Figure 7. 

The frequency domain-converted audio signals are processed in a symbol 
25 values derivation function 1 10, to produce a stream of symbol values for each code 
symbol included in the received audio signal. The produced symbol values may 
represent, for example, signal energy, power, sound pressure level, amplitude, etc., 
measured instantaneously or over a period of time, on an absolute or relative scale, 
and may be expressed as a single value or as multiple values. Where the symbols are 
30 encoded as groups of single frequency components each having a predetermined 

frequency, the symbol values preferably represent either single frequency component 
values or one or more values based on single frequency component values. 
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The function 1 10 may be carried out by a digital processor, such as a digital 
signal processor (DSP) which advantageously carries out some or all of the other 
functions of decoder 100. However, the function 1 10 may also be carried out by an 
application specific integrated circuit, or by any other suitable device or combination 

5 of devices, and may be implemented by apparatus apart from the means which 
implement the remaining functions of the decoder 100. 

The stream of symbol values produced by the function 1 10 are accumulated 
over time in an appropriate storage device on a symbol-by-symbol basis, as indicated 
by the function 116. In particular, the function 1 16 is advantageous for use in 

10 decoding encoded symbols which repeat periodically, by periodically accumulating 
symbol values for the various possible symbols. For example, if a given symbol is 
expected to recur every X seconds, the function 1 16 may serve to store a stream of 
symbol values for a period of nX seconds (n>l), and add to the stored values of one or 
more symbol value streams of nX seconds duration, so that peak symbol values 

1 5 accumulate over time, improving the signal-to-noise ratio of the stored values. 

The function 1 16 may be carried out by a digital processor, such as a DSP, 
which advantageously carries out some or all of the other functions of decoder 100. 
However, the function 1 10 may also be carried out using a memory device separate 
from such a processor, or by an application specific integrated circuit, or by any other 

20 suitable device or combination of devices, and may be implemented by apparatus 
apart from the means which implements the remaining functions of the decoder 100. 

The accumulated symbol values stored by the function 1 16 are then examined 
by the function 120 to detect the presence of an encoded message and output the 
detected message at an output 126. The function 120 can be carried out by matching 

25 the stored accumulated values or a processed version of such values, against stored 
patterns, whether by correlation or by another pattern matching technique. However, 
the function 120 advantageously is carried out by examining peak accumulated 
symbol values and their relative timing, to reconstruct their encoded message. This 
function may be carried out after the first stream of symbol values has been stored by 

30 the function 1 16 and/or after each subsequent stream has been added thereto, so that 
the message is detected once the signal-to-noise ratios of the stored, accumulated 
streams of symbol values reveal a valid message pattern. 
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Figure 8 is a flow chart for a decoder according to one advantageous 
embodiment of the invention implemented by means of a DSP. Step 130 is provided 
for those applications in which the encoded audio signal is received in analog form, 
for example, where it has been picked up by a microphone (as in Figure 6 

5 embodiment) or an RF receiver. 

The decoder of Figure 8 is particularly well adapted for detecting code 
symbols each of which includes a plurality of predetermined frequency components, 
e.g. ten components, within a frequency range of 1000 Hz to 3000 Hz. It is designed 
specifically to detect a message having the sequence illustrated in Figure 3C wherein 

1 0 each symbol occupies an interval of one half-second. In this exemplary embodiment, 
it is assumed that the symbol set consists of twelve symbols, each having ten 
predetermined frequency components, none of which is shared with any other symbol 
of the symbol set. It will be appreciated that the Figure 8 decoder may readily be 
modified to detect different numbers of code symbols, different numbers of 

1 5 components, different symbol sequences and symbol durations, as well as components 
arranged in different frequency bands. 

In order to separate the various components, the DSP repeatedly carries out 
FFT's on audio signal samples falling within successive, predetermined intervals. 
The intervals may overlap, although this is not required. In an exemplary 

20 embodiment, ten overlapping FFT's are carried out during each second of decoder 
operation. Accordingly, the energy of each symbol period falls within five FFT 
periods. The FFT's may be windowed, although this may be omitted in order to 
simplify the decoder. The samples are stored and, when a sufficient number are thus 
available, a new FFT is performed, as indicated by steps 134 and 138. 

25 In this embodiment, the frequency component values are produced on a 

relative basis. That is, each component value is represented as a signal-to-noise ratio 
(SNR), produced as follows. The energy within each frequency bin of the FFT in 
which a frequency component of any symbol can fall provides the numerator of each 
corresponding SNR. Its denominator is determined as an average of adjacent bin 

30 values. For example, the average of seven of the eight surrounding bin energy values 
may be used, the largest value of the eight being ignored in order to avoid the 
influence of a possible large bin energy value which could result, for example, from 
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an audio signal component in the neighborhood of the code frequency component. 
Also, given that a large energy value could also appear in the code component bin, for 
example, due to noise or an audio signal component, the SNR is appropriately limited. 
In this embodiment, if SNR=>6.0, then SNR is limited to 6.0, although a different 

5 maximum value may be selected. 

The ten SNR's of each FFT and corresponding to each symbol which may be 
present, are combined to form symbol SNR's which are stored in a circular symbol 
SNR buffer, as indicated in step 142 and illustrated schematically in Figure 9. In 
certain embodiments, the ten SNR's for a given symbol are simply added, although 

1 0 other ways of combining the SNR's may be employed. 

As indicated by Figure 9, the symbol SNR's for each of the twelve symbols A, 
B and 0-9, are stored in the symbol SNR buffer as separate sequences, one symbol 
SNR for each FFT for 50 FFT's. After the values produced in the 50 FFT's have been 
stored in the symbol SNR buffer, new symbol SNR's are combined with the 

15 previously stored values, as described below. 

When the symbol SNR buffer is filled, this is detected in a step 146. In certain 
advantageous embodiments, the stored SNR's are adjusted to reduce the influence of 
noise in a step 152, although this step is optional in many applications. In this 
optional step, a noise value is obtained for each symbol (row) in the buffer by 

20 obtaining the average of all stored symbol SNR's in the respective row each time the 
buffer is filled. Then, to compensate for the effects of noise, this average or "noise" 
value is subtracted from each of the stored symbol SNR values in the corresponding 
row. In this manner, a "symbol" appearing only briefly, and thus not a valid 
detection, is averaged out over time. Referring also to Figure 3C, in order to avoid 

25 inflating the noise value at the decoder, preferably the encoding scheme is constrained 
so that the same symbol does not appear twice in the first half of the message (i.e., 
within symbol sequence S A , Si, S2, S3, S4). 

After the symbol SNR's have been adjusted by subtracting the noise level, the 
decoder attempts to recover the message by examining the pattern of maximum SNR 

30 values in the buffer in a step 156. In certain embodiments, the maximum SNR values 
for each symbol are located in a process of successively combining groups of five 
adjacent SNR's, by weighting the values in the sequence in proportion to the 
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sequential weighting (6 10 10 10 6) and then adding the weighted SNR's to produce a 
comparison SNR centered in the time period of the third SNR in the sequence. This 
process is carried out progressively throughout the fifty FFT periods of each symbol. 
For example, a first group of five SNR's for the "A" symbol in FFT periods 1 through 

5 5 are weighted and added to produce a comparison SNR for FFT period 3. Then a 
further comparison SNR is produced using the SNR's from FFT periods 2-6, and so 
on until comparison values have been obtained centered on FFT periods 3 through 48. 
However, other means may be employed for recovering the message. For example, 
either more or less than five SNR f s may be combined, they may be combined without 

10 weighing, or they may be combined in a non-linear fashion. 

After the comparison SNR values have been obtained, the decoder examines 
the comparison SNR values for a message pattern. First, the marker code symbols Sa 
and Sb are located. Once this information is obtained, the decoder attempts to detect 
the peaks of the data symbols. The use of a predetermined offset between each data 

15 symbol in the first segment and the corresponding data symbol in the second segment 
provides a check on the validity of the detected message. That is, if both markers are 
detected and the same offset is observed between each data symbol in the first 
segment and its corresponding data symbol in the second segment, it is highly likely 
that a valid message has been received. 

20 With reference both to Figures 3C and 9, assuming that the beginning of the 

buffer corresponds with the beginning of the message (which usually is not the case), 
a peak P of the comparison SNR's for the "A" symbol should appear in the third FFT 
period, as indicated. Then the decoder will expect the next peak to appear in the 
position corresponding to the first data symbol 0-9 in the eighth FFT period. In this 

25 example, it is assumed that the first data symbol is "3". If the last data symbol is "4 M 
and the value of 8 is 2, the decoder will find a peak of the symbol "6" in FFT period 
48, as indicated in Figure 9. If the message is thus detected (i.e., markers detected 
with data symbols appearing where expected and with the same offset throughout), as 
indicated in steps 162 and 166, the message is logged or output and the SNR buffer is 

30 cleared. 

However, if the message is not thus found, a further fifty overlapping FFT's 
are carried out on the following portions of the audio signal and the symbol SNR's so 
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produced are added to those already in the circular buffer. The noise adjustment 
process is carried out as before and the decoder attempts to detect the message pattern 
again. This process is repeated continuously until a message is detected. In the 
alternative, the process may be carried out a limited number of times. 
5 It will be apparent from the foregoing to modify the operation of the decoder 

depending on the structure of the message, its timing, its signal path, the mode of its 
detection, etc., without departing from the scope of the present invention. For 
example, in place of storing SNR's, FFT results may be stored directly for detecting a 
message. 

1 0 Figure 1 0 is a flow chart for another decoder according to a further 

advantageous embodiment likewise implemented by means of a DSP. The decoder of 
Figure 10 is especially adapted to detect a repeating sequence of five code symbols 
consisting of a marker symbol followed by four data symbols wherein each of the 
code symbols includes a plurality of predetermined frequency components and has a 

1 5 duration of one-half second in the message sequence. It is assumed that each symbol 
is represented by ten unique frequency components and that the symbol set includes 
twelve different symbols A, B and 0-9, as in the code of Figure 3C. However, the 
Figure 9 embodiment may readily be modified to detect any number of symbols, each 
represented by one or more frequency components. 

20 Steps employed in the decoding process illustrated in Figure 1 0 which 

correspond to those of Figure 8 are indicated by the same reference numerals, and 
these steps consequently are not further described. The Figure 10 embodiment uses a 
circular buffer which is twelve symbols wide by 150 FFT periods long. Once the 
buffer has been filled, new symbol SNRs each replace what are than the oldest 

25 symbol SNR values. In effect, the buffer stores a fifteen second window of symbol 
SNR values. 

As indicated in step 174, once the circular buffer is full, its contents are 
examined in a step 178 to detect the presence of the message pattern. Once full, the 
buffer remains full continuously, so that the pattern search of step 178 may be carried 
30 out after every FFT. 

Since each five symbol message repeats every 2 x /i seconds, each symbol 
repeats at intervals of 2Vz seconds or every 25 FFT's. In order to compensate for the 
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effects of burst errors and the like, the SNR's R| through Ri 50 are combined by adding 
corresponding values of the repeating messages to obtain 25 combined SNR values 
SNR n , n=l,2...25, as follows: 

5 

SNR n = SR n+ 25i 

i=0 

Accordingly, if a burst error should result in the loss of a signal interval i, only 
one of the six message intervals will have been lost, and the essential characteristics 
of the combined SNR values are likely to be unaffected by this event. 

Once the combined SNR values have been determined, the decoder detects the 
position of the marker symbol's peak as indicated by the combined SNR values and 
derives the data symbol sequence based on the marker's position and the peak values 
of the data symbols. 

Once the message has thus been formed, as indicated in steps 182 and 183, the 
message is logged. However, unlike the embodiment of Figure 8 the buffer is not 
cleared. Instead, the decoder loads a further set of SNR's in the buffer and continues 
to search for a message. 

As in the decoder of Figure 8, it will be apparent from the foregoing to modify 
the decoder of Figure 10 for different message structures, message timings, signal 
paths, detection modes, etc., without departing from the scope of the present 
invention. For example, the buffer of the Figure 10 embodiment may be replaced by 
any other suitable storage device; the size of the buffer may be varied; the size of the 
SNR values windows may be varied; and/or the symbol repetition time may vary. 
Also, instead of calculating and storing signal SNR's to represent the respective 
symbol values, a measure of each symbol^ value relative to the other possible 
symbols, for example, a ranking of each possible symbol's magnitude, is instead used 
in certain advantageous embodiments. 

In a further variation which is especially useful in audience measurement 
applications, a relatively large number of message intervals are separately stored to 
permit a retrospective analysis of their contents to detect a channel change. In another 
embodiment, multiple buffers are employed, each accumulating data for a different 
number of intervals for use in the decoding method of Figure 8. For example, one 
buffer could store a single message interval, another two accumulated intervals, a 
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third four intervals and a fourth eight intervals. Separate detections based on the 
contents of each buffer are then used to detect a channel change. 

Although illustrative embodiments of the present invention and modifications 
thereof have been described in detail herein, it is to be understood that this invention 
5 is not limited to these precise embodiments and modifications, and that other 

modifications and variations may be effected therein by one skilled in the art without 
departing from the scope and spirit of the invention as defined by the appended 
claims. 
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WHAT IS CLAIMED IS: 

1 . A system for decoding at least one message symbol represented by a 
plurality of code symbols in an audio signal, comprising: 

means for receiving first and second code symbols representing a common 
5 message symbol, the first and second code symbols being displaced in time in the 
audio signal; 

means for accumulating a first signal value representing the first code symbol 
and a second signal value representing the second code symbol; and 

means for examining the accumulated first and second signal values to detect 
1 0 the common message symbol. 

2. The system of claim 1, wherein the accumulating means is operative to 
produce a third signal value derived from the first and second signal values and the 
examining means is operative to detect the common message symbol based on the 
third symbol value. 

15 3. The system of claim 2, wherein the accumulating means is operative to 

produce the third signal value by linearly combining the first and second signal 
values. 

4. The system of claim 2, wherein the accumulating means is operative to 
produce the third signal value as a non-linear function of the first and second signal 

20 values. 

5. The system of claim 2, wherein the first and second code symbols each 
comprise a predetermined number of frequency components, and further comprising 
means for producing first and second sets of component values, each set 
corresponding to a respective one of the first and second code symbols and each 

25 component value of each set representing a characteristic of a respective frequency 
component of the corresponding symbol, and means for producing the first signal 
value based on the first set of component values and producing the second signal 
value based on the second set of component values. 

6. The system of claim 2, wherein the receiving means is operative to 
30 receive plural sets of first and second code signals, each set representing a respective 

one of a plurality of message symbols arranged as a message having a predetermined 
sequence including at least one marker symbol and at least one data symbol, the 
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accumulating means is operative to accumulate sets of first and second signal values, 
each signal value set corresponding to a respective one of the sets of first and second 
code signals and including a first signal value representing the first code signal of the 
respective code signal set and a second signal value representing the second code 
5 signal thereof and the examining means is operative to detect the message by 

detecting the presence of the marker symbol based on its signal value set and to detect 
at least one data symbol based on the detected presence of the marker symbol and the 
corresponding signal value set of the at least one data symbol. 

7. The system of claim 1, wherein the accumulating means is operative to 
1 0 store the first and second signal values, and the examining means is operative to 

detect the common message symbol by examining both of the first and second signal 
values. 

8. The system of claim 7, wherein the accumulating means is operative to 
produce the first and second signal values based on multiple other signal values. 

1 5 9. The system of claim 8, wherein the first and second signal values are 

produced from respective sets of time displaced signal values, each of the time 
displaced signal values representing a value of a respective one of the first and second 
code symbols during a corresponding time period thereof. 

10. The system of claim 8, wherein the first and second code symbols each 
20 comprise a predetermined number of frequency components, and further comprising 

means for producing first and second sets of component values, each set 
corresponding to a respective one of the first and second code symbols and each 
component value of each set representing a characteristic of a respective frequency 
component of the corresponding symbol, and means for producing the first signal 
25 value based on the first set of component values and producing the second signal 
value based on the second set of component values. 

1 1 . The system of claim 1, wherein the receiving means comprises an 
acoustic transducer for transducing an acoustic audio signal to an electrical signal, the 
acoustic audio signal having a plurality of code symbols representing a plurality of 

30 message symbols comprising source data for the acoustic audio signal, and further 
comprising a memory for storing indications of detected message symbols. 
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12. The system of claim 11, further comprising a housing for the system 
adapted to be carried on the person of an audience member and means for transmitting 
the stored data for use in producing audience estimates. 

13. A method for decoding at least one message symbol represented by a 
5 plurality of code symbols in an audio signal, comprising: 

receiving first and second code symbols representing a common message 
symbol, the first and second code symbols being displaced in time in the audio signal; 

accumulating a first signal value representing the first code symbol and a 
second signal value representing the second code symbol; and 
1 0 examining the accumulated first and second signal values to detect the 

common message symbol. 

14. The method of claim 13, wherein the step of receiving first and second 
code symbols comprises transducing an acoustic audio signal to an electrical signal, 
the acoustic audio signal having a plurality of message symbols comprising source 

1 5 data for the acoustic audio signal, and further comprising storing data representing 
indications of detected message symbols. 

15. The method of claim 14, further comprising transmitting the stored 
data for use in producing audience estimates. 

16. A system for decoding at least one message symbol represented by a 
20 plurality of code symbols in an audio signal, comprising: 

an input device for receiving first and second code symbols representing a 
common message symbol, the first and second code symbols being displaced in time 
in the audio signal; and 

a digital processor in communication with the input device to receive data 
25 therefrom representing the first and second code symbols, the digital processor being 
programmed to accumulate a first signal value representing the first code symbol and 
a second signal value representing the second code symbol, the digital processor 
being further programmed to examine the accumulated first and second signal values 
to detect the common message symbol. 
30 17. The system of claim 16 wherein the input device comprises an acoustic 

transducer for transducing an acoustic audio signal to an electrical signal, the acoustic 
audio signal having a plurality of code symbols representing a plurality of message 
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symbols comprising source data for the acoustic audio signal, the digital processor 
having a memory for storing data representing indications of detected message 
symbols. 

1 8. The system of claim 1 7, further comprising a housing for the system 
5 adapted to be carried on the person of an audience member and means for transmitting 
the stored data for use in producing audience estimates. 
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